What is the purpose of this change ?

What is the purpose of this change ?

This would be so that an artifact can publish exclusively for dotc, but still be consumed from scala 2

This would be so that an artifact can publish exclusively for dotc, but still be consumed from scala 2

Given this limitation:

Implementation of the macro cannot use Scala 2 macros (like reify { } and q"...")

It seems like very few macros could actually take advantage of this.

Wouldn't it be much simpler to get the build tool to put classfiles compiled with Scala 2 in Dotty artifacts instead ?

Given this limitation:

Implementation of the macro cannot use Scala 2 macros (like reify { } and q"...")

It seems like very few macros could actually take advantage of this.

Wouldn't it be much simpler to get the build tool to put classfiles compiled with Scala 2 in Dotty artifacts instead ?

so scala 2 compiled macro is on classpath to dotc, symbol to scala 2 macro impl is on the rhs of a macro def in dotty, and then TASTy links to that symbol?

so scala 2 compiled macro is on classpath to dotc, symbol to scala 2 macro impl is on the rhs of a macro def in dotty, and then TASTy links to that symbol?

I don't understand what this would allow exactly.

Here's a quick sketch of what I have in mind:

• If you have a Dotty project and need to write some Scala 2 macros, put them in .scala2 files
• When sbt compiles a dotty project, it will first compile the regular .scala files with dotty, then compile the .scala2 files using scalac with the result of the previous compilation round on the classpath.
• The classfiles generated from the .scala2 files should go in a scala-2/ directory in the jar (similar to how you can put java 9 files in a 9/ subdirectory: https://openjdk.java.net/jeps/238)
• scalac should learn to go look for classfiles in the scala-2/ directory of a jar (other compilers should ignore it since package names cannot contain -)

This allow presenting the same user API for Scala 2 and Scala 3 macros, and does not require any change to dotty or tasty.

Alternatively, we could also output the classfiles from .scala2 files next to the regular classfiles, this would not allow the same user API for Scala 2 and 3 (the class names would have to be different) but would be simpler to implement, and would allow using scala 2 files in dotty projects for other reasons. This should be implementable in sbt-dotty without touching sbt or scalac.

Yeah I think the classfiles should live next to each other, with the Scala 2 ones being under Scala 2's current schema, and the new Scala 3 ones adding some variant (let's say HelloMacros_3.class, but I'm guessing that's illegal). That way current 2.13 nsc can continue to dispatch how it's always done when macro expanding and dotc can dispatch with the variant.

It would be good to get a survey of how many macros out there are using reify and q. And even so, how difficult is it to migrate off them? Could it be somewhat automated? Can we do a macro expansion-based rewrite of them?

Lastly, the strategy of using nsc after dotc requires depending/using nsc, which includes paying the price of spinning it up. It would be nice to create a library jar in 1 dotc run only.

If a Scala 2 macro uses q"..." or reify, this is how I would see things being done:

• Have a separate configuration Scala2Macros which contains the implementation of the Scala 2 macros, compiled with Scala 2.
• The main Compile configuration is compiled with Scala 3, depends on the Scala2Macros configuration, and contains the definitions of the Scala 2 macros, which call the implementations in the separate compilation. It can also contain the definition and implementation of Scala 3 macros.

With this architecture, there is no need to teach either sbt or scalac/dotc to deal with weird classfiles (side classfiles or multi-jar whatever).

• The main Compile configuration is compiled with Scala 3, depends on the Scala2Macros configuration...

... no need to teach either sbt ...

If I understand you correctly, you can't make that dependency without teaching sbt.

Hum, it should be a straightforward dependency between configurations, like there already exist between Test and Compile. Can't we add a dependency with a mere setting? Do we really have to go change sbt itself to do that? That sounds very unfortunate.

Yeah, you can't redefine the hierarchy of Configurations, even just appending. Some users want to redefine IntegrationTest to extend Test, but can't.

Well ... in the worst case one can use a separate project then, instead of a configuration.

I've always assumed there'd be bits and bobs to change here and there in the toolchain to facilitate Scala 3, so I'm not concerned about that.

But, ideally, I'd still prefer not warming up two compilers for 1 jar.

This was added in #8811